Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Enable the integration tests for Ollama#802

Merged
rdimitrov merged 3 commits intomainfrom
ollama-integration-tests
Jan 29, 2025
Merged

Enable the integration tests for Ollama#802
rdimitrov merged 3 commits intomainfrom
ollama-integration-tests

Conversation

@rdimitrov
Copy link
Member

@rdimitrov rdimitrov commented Jan 29, 2025 •
edited
Loading

The following PR enables the integration tests for Ollama.

Details:

  • Starts the CodeGate container in the host network (so everything can talk to each other without the need to specify explict ports), we don't have port conflicts anyway
  • Starts the Ollama container and tests if it runs
  • Downloads and runs the qwen2.5-coder:0.5b model
  • Updates the tests to use qwen2.5-coder:0.5b
  • Updates the chat test to expect a simpler welcome message
  • Adds a missing header in the tests

Fixes: #815

@rdimitrov
Add integration tests for Ollama
e217405 
Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
@rdimitrov rdimitrov force-pushed the ollama-integration-tests branch from 1da49fc to e217405 Compare January 29, 2025 07:36
@jhrozek
Copy link
Contributor

jhrozek commented Jan 29, 2025

I think this is a bug in the text that uncovers a bug in codegate:

decoded_line = ' {"error": "[Errno -2] Name or service not known", "type": "error",      │ │
│ │                    "choices": ['+2                                                           │ │
│ │                e = IndexError('list index out of range')                                     │ │
│ │        json_line = {                                                                         │ │
│ │                    │   'error': '[Errno -2] Name or service not known',                      │ │
│ │                    │   'type': 'error',                                                      │ │
│ │                    │   'choices': []                                                         │ │
│ │                    }                                                                         │ │
│ │             line = b'data: {"error": "[Errno -2] Name or service not known", "type":         │ │
│ │                    "error", "choice'+7                                                       │ │

So what happens here is that we get a line and expect it to cointain choices but it only contains an error. We should handle the error, perhaps by raising an HTTP exception. But the root cause is that we cannot reach a service (or ollama can't?)

@rdimitrov
Set the CODEGATE_OLLAMA_URL explicitly
bc8bd40 
Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
@rdimitrov rdimitrov force-pushed the ollama-integration-tests branch from 8a6881f to bc8bd40 Compare January 29, 2025 12:15
@rdimitrov
Copy link
Member Author

rdimitrov commented Jan 29, 2025

I think this is a bug in the text that uncovers a bug in codegate:

decoded_line = ' {"error": "[Errno -2] Name or service not known", "type": "error",      │ │
│ │                    "choices": ['+2                                                           │ │
│ │                e = IndexError('list index out of range')                                     │ │
│ │        json_line = {                                                                         │ │
│ │                    │   'error': '[Errno -2] Name or service not known',                      │ │
│ │                    │   'type': 'error',                                                      │ │
│ │                    │   'choices': []                                                         │ │
│ │                    }                                                                         │ │
│ │             line = b'data: {"error": "[Errno -2] Name or service not known", "type":         │ │
│ │                    "error", "choice'+7                                                       │ │

So what happens here is that we get a line and expect it to cointain choices but it only contains an error. We should handle the error, perhaps by raising an HTTP exception. But the root cause is that we cannot reach a service (or ollama can't?)

Thanks! I'll file a separate issue for this and fix it there.

@rdimitrov rdimitrov mentioned this pull request Jan 29, 2025
Open
@rdimitrov rdimitrov changed the title Add integration tests for Ollama Enable the integration tests for Ollama Jan 29, 2025
jhrozek
jhrozek approved these changes Jan 29, 2025
View reviewed changes
Copy link
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yay!

@rdimitrov rdimitrov merged commit d897f7e into main Jan 29, 2025
4 checks passed
@rdimitrov rdimitrov deleted the ollama-integration-tests branch January 29, 2025 14:07
lukehinds pushed a commit that referenced this pull request Jan 31, 2025
@rdimitrov @lukehinds
Enable the integration tests for Ollama (#802)
e064dac 
* Add integration tests for Ollama

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>

* Set the CODEGATE_OLLAMA_URL explicitly

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>

---------

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@jhrozek jhrozek jhrozek approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Enable the integration tests for Ollama

2 participants

@rdimitrov @jhrozek